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1.  Introduction.  Let  W(t),  t  >  0,  W(0)  •  0  be  a  standard  Wiener 

process,  independent  of  N(t),  t  >  0,  N(0)  ■  0,  a  Poisson  process  with 

(constant)  unit  jumps,  and  EN(t)  -  \t,  t  >  0.  Let  their  sigma  field*  be 

F(t)  -  a(W(s),0  <  s  <  t)  and  G(t)  •  a(N(s),0  ^  s  <  t) ,  respectively.  Let 

X(t)  be  a  stochastic  process  that  (for  ^  a  X  )  satisfies  the  Ito  stochastic 

Oc  c 

differential  equation 

(1.1)  Xc(t)  -  u(X(t))dt  +  dW(t)  +  dN( t) 

X(0)  -  x  , 

where  x  is  real,  and  u(X(t))  is  measurable  with  respect  to  cr(F(t)  U  G(t)) 
(i.e.  u  is  non-anticipative)  and  satisfies,  for  A,  B  constants,  B  >  0, 
and  )a|  <  B, 

(1.2)  ju-A|  <  B 

for  all  0  <  t  <  T,  0  <  T  <  «  a  constant.  Cne  cost  function  for  a  given 
u  satisfying  (1.2)  is,  for  a  >  0  a  constant,  and  qp(x)  a  symmetric  positive, 
increasing  on  the  positive  x-axis  function  of  polynomial  growth  as  x  -*  •, 
that  is,  for  some  3  >  0, 

(1.3)  sp(x)/|x}^  -  0  as  J x j  -  <p(x)  -  •,  and  9XX(*)-  •  as  x  - 
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(1.4)  J(u)  -  J1  e~asE(<j)(x(s))  +  |u(X(s)|)ds. 

Another  cost  function  to  be  considered  is 

(1.5)  K(u)  »  (Va*B(<p(X(a))  +  u2(X(s)))ds. 

J0 

This  latter  cost  function  will  be  briefly  treated  in  section  4. 

The  object  is  to  characterize  the  optimal  u  for  which  J  or  K 
is  minimized,  respectively.  The  cases  T  <  <*  and  T  «  •  are  treated 
separately.  The  existence  of  an  optimal  u  depends  an  assumptions  about 
the  asymptotic  behavior  of  certain  partial  differential-difference  equ¬ 
ations.  The  method  employs  a  suitable  Bellman  equation,  a  maximum 
principle  for  parabolic  partial  differential-difference  equations  and 
the  Ito  rule.  The  method  follows  [4]. 


2.  Finite  Interval  Control. 


Let  T  <  «*.  Define,  for  0  t  <  T, 


(2.1)  V  -  V(x,c>  -  inf  Je‘orSE(tp(X2(s))  +  |u(X(s»})ds 

Ju-A  |<  B  *0 


V(x,0)  *  x 


~t  rh  -t+h  pt+h 

Writing  a  j  +  -  ,  heuristic  arguments  (or  see 

J0  J0  Jh  Jt 

(21,  pp.  179-180)  yield  a  Bellman  equation  (where  V  aV(x,t), 

v .  f , »  -  4 . « .  .<*» 

x  dx  XX  3x2  _ 


Accession  For 


(2.2)  <p(x)  +  inf  (uV  +|o|)  +  £  V  -  «V-V  NTis 

\  I  I  ^  n  X  ^  XX  C 


|u-a|<b 


+  \(V(x+l,t)-V)  -  0. 


Cip. 

^  tfin 

V  *  > 


DTIO  T'!< 

Unsu)vo-.;nc£>d 
J'.‘  '•  1 1 .  i  o  :  t  i  on _ 

By - - - 

Distribution/ 
Availability  Codas 
JAvail  and/or  ~ 
Dist  Special 


□  C 


Define 


(B-A)(l-a) 

0 


(2.3)  g(a)  a  inf  (ua+juj)  =■ 
|u-A|<B 


(A+B)(14a) 


if  a  >  1 
if  |a{  <  1 
if  a  <  -  1 


Then  (2.2),  (2.3)  become 

(2.4)  9  +  g(Vx)  +  j  Vxx  -  oV-Vt  +  X(V(x+l,t)-<0  -  0. 

On  heuristic  grounds,  a  solution  to  (2.4)  is  sought  such  that  for 
functions  b^(t)  <  b2(t);  0  <  t  <  T  to  be  determined, 

(2.5a)  9  +  (A-*)(1+Vx)  +  j  V^-oV-V,. 

+  \(V(x+l,t)-V)  -  0 

for  x  <  b^(t)  , 

(2.5b)  9  +  \  -  aV-Vt  +  x(V(x*l,t)-V)  -  0 

for  b^(t)  <  x  <  b2(t), 

and 

(2.5c)  9  +  (B-A)(l-Vx)  +  i  Vxx  -  aV-Vfc  +  X(V(xH,t)-V)  -  0 

for  x  >  b2  ^  * 

The  functions  b^(t),  b^(t)  are  to  be  obtained  from  these  matching 
conditions,  where  for  0  <  t  <  T, 

V  a  Vl  in  (2.5a),  V  a  V2  in  (2.5b),  V  a  V3  in  (2.5c): 

Vi(b1(t)’t)  -  v2(b1(t),t) 

(2.6) 


V,(b,(t),t)  -  V.(b,(t),t) 


Vl,x(bl(t)’C>  -1 

V2,x(b2(t)’c)  "  V3,x(b2(t)'t)  “  +1- 

(2.7)  V^x.O)  -  V2(X,0)  -  V3(X,0)  -  0. 

(2*7a)  ^  0 

(2‘7b)  V2,xx(bl(t),t)  ^  °* 

For  R  a  constant,  denote 

(2.3)  J(x,t,R)  a  J  e”eS[”E(cp(Rs+W(s)-m(s)+x)+|R| Ids. 

0 

It  may  be  verified  that  J(x,t,A+B)  is  a  particular  solution  to 
(2.5a),  that  J(x,t,0)  is  a  particular  solution  to  (2.5b)  and  that 
J(x,t,A-B)  is  a  particular  solution  to  (2.5c).  The  solutions  to  (2. 
will  be  shown  to  follow  if  this  condition  holds. 


Assumption  1.  There  i3  a  non-zero  solution,  for  each  t. 


H1(x,t)  with  H1(x,0)  -  0  to  (HSI(x,t)) 


(2.9)  (A+B)(l-Hix)  +  \  -  aH-Ht  +  X(H(x«.,t)-H)  -  0 

such  that 


(2.10) 


(*,t)  -  0(e+TX) 

Y,«(x-t)  ■ 0<e+ct) 


for  some  r>0,  L  >  0  ,  as  x  -  •  *. 

Also,  there  is  a  non-zero  solution  Hj(x,t)  with  ^(x.O) 


0  to 


(2.  H)  y  -  aH-Ht  +  X(H(x+l , t)-H)  -  0. 

Further,  there  is  a  non-zero  solution  Hj(x,t)  with  H^x.O)  ■  0 


(2.12) 


(3-A)(l-Hx) 


+  r  H 
2  xx 


-  aH-H  +  X(H(x+l,t)-H)  =»  0 


5a-c) 


to 
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such  that 

(2.13)  H3(x,t)  *  0(e"kx) 

H3,xx(x’t)  *  °<e“CX> 

for  some  k  >  0,  c  >  0,  all  t,  »s  x  t  +  », 

If  Che  above  Assumption  1  holds  Chen  let 

<2. 14a)  V^x.C)  aJ(x,t,4+B)  +  H^x.C) 

(2.14b)  V2(x,t)  a  J(x,t,0)  +  H2(x,t) 

(2.14c)  V3(x,t)  a  J(x,C,A-B)  +  H3(x,t). 

AssumpCion  2 .  The  V^(x.t),  1  <  £  <  3  of  (2.l4a-c)  which  satisfy 
(2.5a-c)  respectively,  and  conditions  (2.6).  (2.7),  determine  b^(t)  <b2(t). 
This  motivates 

Theorem  1.  If  the  conditions  of  section  2  and  Assumptions  1  and  2 

hold  for  0  <  t  <  T  <  •,  then  the  optimal  may  be  expressed  in  closed 

loop  form  as 

IA+B  if  XQ(t)  <  bx(T-t) 

0  if  b^T-t)  <  XQ(t)  <  b2(T-t) 

A-B  if  XQ(C)  >  b2(T-t) 

where 

(2.16)  dXQ(t)  -  vVr>'dC  +  dW(C)  +  dN (t) 

XQ(0)  -  x. 

Proof.  Let  D  »  (omitting  (x,t)  arguments). 

Claim.  D  >  0  all  (x,t). 

Proof  of  Claim.  From  (2.4),  omitting  (x,t),let 
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(2.17)  L(D)  =  g(Dx)  +  IDXx'(a+X)D‘Dt  * 

From  (1.3),  the  conditions  on  9,  end  (2.8)  -  (2.14),  it  follows 
that,  for  each  t, 

(2.18)  D  >  0  as  }x)  -»  ®. 

Suppose  that  there  is  an  r  >  and  a  Y,  0  <  y  <  1  for  fixed  t,  such 

that 

(2.19)  D(x)  <  0  b2  <  r-y  <  x  <  r 
(2.19a)  D(r)  -  0 

(2.20)  D(x)  >0,  x  >  r. 

We  now  obtain  a  contradiction  to  (2.19). 

From  (1.3),  9  >0,  hence  (2.17)  and  (2.20)  imply  that 

'XX 

L(D)  <0,  x  >  r-1. 

It  follows  from  a  maximum  principle  (Lemma  I (after  multiplying 
by  -l)[l],  p-  34)  that  D  cannot  have  a  negative  minimum  for  the  fixed 
t,  for  x  >  r-y.  From  this  and  (2.7a),  (2. 18) , (2. 20) ,  if  follows  that  if  D 
were  negative  for  any  x  >  r-y  ,  it  would  have  a  negative  minimum,  which 
is  not  allowed  by  the  maximum  principle.  Hence  D  >  0  for  x  >  r-y, 

contradicting  (2.19)  and  completing  the  claim  for  x>  b^. 

For  x  <  b^,  a  similar  argument  using  (2.7b),  (2.18)  yields  that  D(x)  >  0 
for  b2-fi  <  x  <  b2  for  appropriate  0  <  6  <  1.  Continuing  the  argument  by 
iteration  yields  that  D(x)  >  0  all  x,  for  each  t. 

The  claim  implies  that  is  increasing  in  x  for  each  t  and 
hence  that  (2.5)-(2.14)  indeed  yields  a  solution  to  the  Bellman  equation 
(2.4). 

To  show  Uq  is  optimal,  define,  for  0  <  t  <  T 

(2.21)  X(X(t),t)  s  V(X(t),T-t)e_ott. 

I 
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Noting  that  K(X(0) ,0)  -  V(x,T)  and  K(X(T),T)  »  0,  the  Ito  rule 
([2] ,pp. 125-126)  applied  to  (2.21)  for  an  admissible  u  and  its  corres¬ 
ponding  X(t)  yields,  upon  integrating  from  0  to  T,  and  adding  and  sub¬ 
tracting  appropriate  terms ,  that 


(2.22)  J1 


e'a3(<p(X(s))  +  |u(X(s))|)ds  -  V (x,X) 


|Va%(X(«))  +  g(V  (X(s),X-s))  +T  V  (X(s),T-s) 

v  A  *  XX 


-aV(X(s),T-s)  -  Vt(X(s),T-s))ds 


+  I  e*aSV(X(s),X-s)dN(s) 
J0 


+  |V- 


,  -  V  (X(s),X-s)dW(s) 
J0  x 


+  j  e_a3(u(X(s))V  (X(s),X-s)  +  |u(X(s)|-g(V  (X(s) ,X-s)))ds. 

J0 

Xhe  fourth  integral  on  the  right  side  of  (2.22)  is  non-negative. 
Upon  taking  expectations  in  (2.22),  the  third  integral  on  the  right 
becomes  zero. 

On  combining  the  first  and  second  integrals  on  the  right  after 
taking  expectations,  and  suppressing  the  X(s)  and  the  (X(s),X-s) 
arguments,  one  obtains  from  (2.22), 


(2.23)  j  e"aSE(<p +|u|)ds  -  V(x,X)  - 


jV0,E(**g(Vx)  +  J  Vxx  -  aV-Vt  +  \(V(X(s)+l,X-s)-V))ds 
f e "a*E (uV  +  J  u | -g (V  ) )  ds . 


The  first  integral  on  the  right  of  (2.23)  is  zero  by  (2.4)  and  the 
second  integral  on  the  right  is  non-negative  by  definition  of  g  in 


(2.3),  with  equality  if  u=Ug.  Hence  from  (2.23), 

(2.24)  f e~aSE(cp+  JuJds  >  V(x,T) 

0 

with  equality  if  u*Uq,  showing  that  u^  is  optimal.  This  completes 
Theorem  1. 

3.  Infinite  Interval  Control.  Assume  that  the  conditions  of  section  2 
hold  and  let  T  »  «•  .  The  cost  function  is  then 

r® 

(3.1)  J(u)  -  J^e‘a3E(<p(X(s»  +  |u(X(s))|)ds 

which  is  finite  by  (1.3)  for  admissible  u. 

Define  V  a  V(x)  as 


(3.2)  V(x)  -  inf  J  e"asE(cp(X(s))  +  ju(X(s)J)ds 
|u-A|<B  0 


where  X(0)  ■  x,  a  constant. 

By  writing  a  +  ,  heuristic  arguments  (see[2],  pp.  179-180), 

0  J0  \ 

yield  a  Bellman  equation,  using  the  abbreviated  arguments  in  (2.2), 


(3.3)  ?(x)  +  inf  (uV  +juj)  +rV  -  a7  +  \(V(xH)-V)  -  0. 

)u-A|<B  x 


i 


As  itt  (2.3)  ,  (2.4)  , 


(3.4)  cp(x)  +  g(Vx)  +  J  -  oV  +  \(V(x+l)-V)  -  0 


On  heuristic  grounds,  a  solution  to  (3.4)  is  sought  such  that  for 
numbers  <  b^  Co  be  determined  below,  omitting  x  arguments, 

(3.5a)  ?  +  (A+B)(l+Vx)  +  J  ^  +  X(V(x+ 1)-V)  -  0 

for  x  <  bj. 


(3.5b)  <?  +  j  -  aV  +  X(V(x+l)-V)  -  0 


for  <  x  <  b2  , 


(3.5c)  <p  +  (B-A)(l-Vx)  +  j  -  aV  +  X(V(xfl)-V)  -  0 


for  x  >  b^. 


The  b^  <  b^  are  to  be  determined  from  the  following  matching  con¬ 
ditions,  where  V  a  Vj  in  (3.5a),  V  a  in  (3.5b)  and  V  a  in  (3.5c). 

w  -vv 


(3.6) 


(3.6a) 


v2(b2)  -v3(b2) 


Vl,x<V  "W*  -1 


V2,x<V  "  V3,x<b2)a  +1- 


V3,xx<V  *  0 

V2,xx(bl>  *  °- 


For  R  a  constant,  denote 


r® 

(3.7)  J(x,R)  s  J  e'aSE(9(Rs+W(s)-W(s)+x)+jR|)ds. 


It  say  be  verified  that  J(x,A+B)  is  a  particular  solution  to  (3.5a), 


that  Jfx.0)  Ls  a  particular  solution  to  (3.5b)  and  J(x,A-B)  is  a 


particular  solution  to  (3.5c). 


Assumption  3. 

There  is  a  non-zero  solution  H^(x)  to 

aent) 

(3.8.) 

(A+B)(l+Hx)  +  ~  -  aH  +  X(H(x+1)-H)  ■ 

i  that 

(3.9) 

Hx(x)  -  0(6^) 

H1,=W  ‘  °<'+VX) 

some  u  >  0,  v 

>  0,  as  x 

There  is  a  non-zero  solution  (x)  to 

(3.10) 

2  Hxx  "  °H  +  X(a(*H)-H)  =  0. 

There  is  a  non-zero  solution  H^x)  to 

(3.11) 

(B-A)(l-H  )  +  \  H  -  oH  +  x(H(xH-l)-H) 

X  Sm  XX 

such  that 

• 

(3.12) 

H3(x)  -  0(e“qX) 

-  °<e'PX) 

£or  some  p  > 

0,  q  >  0  as  x  ■>  +  »  , 

Now  one  set 3 

(3.13a) 

VL(x)  -  J(x,A43)  +  Hl(x) 

(3.13b) 

V2(x)  -  J(x,0)  +  H2(x) 

(3.13c) 

V  (x)  -  J(x,A-B)  +  H_ (x)  . 
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Assumption  4. 


(3.14)  The  b^  <  b2  are  determined  by  the  V^(x),  *  ^  &  £.  ^ 
of  (3.13  a-c). 

Theorem  2.  Under  the  assumptions  of  this  section,  the  optimal 


u»u^  may  be  expressed  as 


(3.15)  u^X^t)) 


A+B  if  X^t)  <  bx 
0  if  bx  <  X^t)  <  b2 
A-B  if  X1(t)  >  b2 


where 


(3.16)  dX1(t)  -  u1(X1(t))dt  +  dW(t)  +  dN(t) 

X^O)  -  x. 

proof.  Let  D  -  V  ,  suppressing  the  x-arguments. 

Claim.  D  >  0  all  x. 

Proof  of  Claim.  From  (3.4) 

(3.17)  K(D)  a  g  (Dx)  +  J  -(o+l)D  -  -y^-WCicH)  . 

By  an  argument  identical  to  that  given  in  the  pooof  of  Theorem  1, 
using  the  appropriate  maximum  principle  (fij.  Theorem  18,  p.  53),  it 
follows  that  D(x)  >  0  for  all  x. 


The  claim  implies  that  is  increasing  in  x  and  hence  that 

(3 . 5 ) — (3 . 14)  yields  a  solution  to  (3.4). 

To  show  u.  is  optimal,  define,  fov  t  >  0, 


(3.18) 


R(X(t))  s  V (X(t) )e"a  . 


Noting  that  R(0)  -  V(x),  the  Ito  rule  ([2],  pp.  125-126)  applied  to 
R(X(t))  followed  by  integration  and  adding  and  subtracting  appropriate 
terms  yields  (arguments  on  the  right  side  not  indicated  are  X(s)) 


(3.19) 


(<p(X(s))  +  |u(X(s))|)ds  +  V(X(t))e‘at-V(x)  - 


(<P  +  S(VX)  +  \  V}Q£-aV)ds  + 


-orSVdN  (s) 


+  J«‘a9VW(s)  +  J  e‘oS(<p+|u|  -g(V  ))ds. 

The  fourth  integral  on  the  right  is  non-negative  by  the  definition  of 
g(x).  Upon  taking  expectations  in  (3.19),  the  third  integral  on  the 
right  vanishes,  and  the  first  and  second  terms  on  the  right  may  be 
combined  to  obtain  (again  the  arguments  not  indicated  are  X(s)) 


(3.20)  J  e“aSE(<p+  JuJ)ds  +  e-atEV(X(t))-V(x)  - 

J0 

f  e~aSE(<p  +  g(V  )+I  V  -aV  +  \(V(X(s)+l)-V))ds 

V  Q  A  Z  XX 

+  |u|  -  g(Vx))ds. 


By  (3.4),  the  first  integral  on  the  right  of  (3.20)  is  zero,  and  the 
second  Integral  on  the  right  is  non-negative  by  the  definition  of  g. 
From  (1.3),  (3.5)-(3.14),  and  the  bounds  on  |u|,  it  follows  that  there 
is  a  constant  L  >  0  such  that  for  all  t, 

(3.21)  E(V(X(t)))e‘ot  £  Le”at 
letting  t  -  <»  in  (3.20),  and  using  (3.21)  one  obtains 


(3.22)  f  e'USE(cp+  }uj)ds  >  V(x) 

J0 


and  (3.22)  J^e"aSE (3))  +  |Ul (Xj (s)) ) )ds  »  V(x) 

so  chat  (3.22)  ,  (3.23)  yield  that  u^  is  optimal,  completing  Theorem  2. 

4.  Alternate  Cost  Function.  The  same  models  as  in  sections  1-3  with 
certain  other  cost  functions  may  be  treated  in  a  similar  way.  For 
example,  the  cost  function  (1.5)  yields,  for  T  <  «•  ,  a  Bellman  equation 
(the  (x,t)  arguments  are  omitted) 

(4.1)  9  +  h(Vx)  +  i  -aV-Vt  +  \(V(x+l,t)-V)  -  0 


where 


(A-HB)(a+A+B)  if  -  |  >  A+B 


(4.2)  h(a) 


inf  (ua+u  ) 
I  u-A  I  <  B 


if  A-B  <  -  !<A+B 


(A-B)  (a+A-B)  if  -  |  <  A-B 


and  from  (4.2), 


(4.3)  u  -  A-B  if  -  |  <  A-B;  u  =-  -  |  if  A-B  <  -  -  <  A+B; 


u  -  A+B  if  -  |  >  A+B. 


One  seeks  a  solution  to  (4.1)  of  the  form 


(4.4a)  9+  (A+B)(Vx+A  +  B)  +  \  Vxx"°V"Vt+  “  0 

if  x  <  k^t) 


(4.4b)  9  -  ~(Vx)2  +  \  -  aV-Vt  +  \(V(*+l,t)-V)  -  0 


if  kx(t)  <  x  <  k2(t) 


1 


and 
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(4.4c)  9  +  (A-B)(V+A-B)  +  |  V^-  aV-Vfc  +  \(V(x+l,t)-V)  -  0 

If  x  >  k2(t). 

The  k^(t),  k2(t)  are  to  be  determined  by,  for  V  a  in  (4.4a), 

V  a  V2  in  (4.4b),  V  a  V3  in  (4.4c),  for  0  £  t  <  T, 

v1(k1(t),t)  -  v2(k1(t),t) 

V2(k2(t),t)  -  v3(k2(t),t) 

(4.5) 

VifX(kl(t),t)  -  V2x(k1(t),t)  -  -  2(A+B) 

V2,x(k2(t),t)  “  v3>x0t2(t),t)  -  -2(A-B). 

and 

(4.6)  V^x.O)  -  V2(x,0)  -  V3(x,0)  -  0  all  x. 

(4.6a)  V2,xx(kl(t),t)  -  ° 

V3,xx(k2(t),t>  -  °* 

The  maximum  principle  ([1],  Lemma  1  p.  34)  may  be  applied  to  (4.4a), 
(4.4c)  as  in  Theorem  1,  as  these  are  linear  in  the  term.  Similarly, 
for  R  a  constant,  if  one  defines 

(4.7)  L(x,t,R)  s  re"0,SE(m(Rs+W(s)-m(s)+x)4R2)ds  , 

J0 


then  L(x,A+B)  is  a  particular  solution  to  (4.4a)  and  L(x,A-B)  is  a 
particular  solution  to  (4.4c).  Adding  an  assumption  similar  to 


assumptions  1  and  2  in  section  2 ,  and  on  the  boundedness  of  solutions 
to  (4.4b),  one  may  obtain  the  optimal  u^  for  this  problem  implicitly 
in  the  form  _ 

A+B  if  ^CO^k^T-t) 

(4.8)  u3(X3(C),t)  -  -|vx(X3(t),t)  if  k1(T-t)<X3(t)  <k2(T 

.  A-B  if  X^(t)  >k2<T-t) , 

where 

dX^t)  -  u3(X3(t))dt  +  dN(t)  +  dW(t), 
using  arguments  as  in  Theorem  1. 

The  case  T  »  •  parallels  that  of  section  3.  The  Bellman  equation 
is,  for  V  a  V (x) , 

(4.9)  <p+h(Vx)  +  i-  -  orV  +  X(V(xhl)-V)  -  0 
and  a  solution  to  (4.9)  is  sought  of  the  form 

(4.10a)  9  +  (A+B)  (Vx+  A+B)  +  |  V^-aV  +  X(V(x+l)-V)  -  0 

for  x  < 

(4.10b)  «p- i(Vx2)  +  j  V^.  aV  +  X(V(x+l)-V)  -  0 

for  <  x  <  , 

and 

(4.10c)  9  +  (A-B)(Vx+A-B)  +  2  V^-  aV  +  X(V(x+l)-V)  -  0 

for  x  > 

where  the  constants  are  to  be  determined  from  the  matching 

conditions  where  V  s  »1  in  (4.10a),  V  a  V2  in  (4.10b),  V  a  V3  in  (4.10c) 


1 
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If 


(4.12) 


(4.11a) 


w 

-  Vjdj) 

vv 

-  W 

"  v2  (Jtj)  .  - 2  (A-HS) 

Y*<V 

’  V3,»<J2)  * 

V2,xx<*l>  ^  0 


(4.12)  L(x,R)  3  J  e"OS(E({p(Rs+W(s)4W(s)+*)4fl2)da, 


it  may  be  shown  chac  L(x,A+B)  is  a  particular  solution  to  (4.10a)  and 
L(x,A-B)  is  a  particular  solution  to  (4.10c).  Adding  an  appropriate 

assumption  similar  to  that  in  section  3,  and  on  the  boundedness  of 
solutions  to  (4.10b),  the  optimal  u^  is  implicitly  expressed  as 


(4.13)  u4(X4(t))  -  < 


A-H3 


if  X4(t)  £  ^ 
-|-Vx(X4(t))  if  £1<X4(t)<£2 


a-b 


if  *4  00  > 


where 

dX4(t)  -  u4(X4(t))dt  +  dW(t)  +  dN (t) 
X4(0)  -  x  . 

5.  Additional  Constraints. 


Certain  additional  constraints  may  be  incorporated  and  treated 
by  those  methods.  For  example,  in  the  case  T  <  «  of  section  2,  the  added 
constraint 


i 
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(5.1)  E(cp(X(a))  +  |u(X(a))  |)  -  C, 

where  a  is  a  constant,  0  <  a  <  T  and  C  >  0,  may  be  incorporated  by 
adding  the  condition 

Vt’(x,t)|  -  e'oaC 

t*a 

to  the  conditions  (2. 6), (2. 7),  and  proceeding  as  before.  See  [3]  for 
another  approach. 

6.  Extensions. 

The  method  applies  to  a  variant  of  the  stochastic  differential 
equation  (1.1).  Let,  for  M  0  a  constant, 

(6.1)  Xt(t)  -  (0X(t)+u(X(t))dt  +  dW(t)+d»(t), 

X(0)  -  x, 

with  control  u(X(t))  satisfying  (1.2)  as  before.  Similarly,  the  cost 
function  J(u)  is  as  in  (1.4). 

The  appropriate  Bellman  equation  for  T  <  °»  is,  where  g  is  as 
in  (2.3), 

(6.2)  0  -  q>(x)-htVx+g(Vx)+  |  V^-V^XWxHW-aV. 

A  solution  of  the  form  (2. 5)- (2. 7)  is  sought  as  before.  To  obtain 
a  particular  solution  to,  e.g.,  the  Bellman  equation 

(6.3)  0  -  cp(x)+xVx+(A+B)  (1+Vt)+  fvxx-aV-Vt 


+X(V(x+l,t)-V)  -  0, 


denote 


Then 


(6.4)  Y(t)  -  e"PtX(t). 

(6.5)  dY(C)  »  e"pt(A+B)dt  +  e“ptdW(t) 

+  e‘ptdH(t) 


Hence,  integrating  (6.5) 

(6.6)  Y(t)  -  x  +  J* e"psdW(s) 


+  fe‘PsdN(s)  +  (A+B)  e**psds 

Jo  J0 


or 


(6 


.7)  X(t)  -  xePt+  ^Si(ept-l)  +  (t“9)dW(») 


"0 


+  f  eP(C's)dN(s). 
J0 


It  may  now  be  verified  that 


(6.8)  J(x,t,A+fl)  aE  f  (q>(xePt+ (ePt-l)  +  fe?(t*s)dW(s) 

J0  P  Jn 


£ 


P(C-s) 


dN(s))  +  (A+B))ds 


is  a  particular  solution  to  (6.3).  The  rest  of  the  construction  and 
matching  and  initial  conditions  and  proofs  are  as  in  section  2. 
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